This year the annual re:invent conference organized by AWS was virtual, free and three weeks long. During multiple keynotes and sessions, AWS announced new features, improvements and cloud services. Below is a review of the main announcements impacting compute, database, storage, networking, machine learning and development.
Compute
On the very first day of the conference, Amazon announced EC2 Mac instances for macOS, adding after many years a new operating system to EC2. This is mainly targeted at processes that only run on Mac OS, like building and testing applications for iOS, MacOS, tvOS and Safari. The first part of Andy Jassy's keynote was focused on announcements related to compute options and serverless technologies. AWS introduced new instance types on different processors and EC2 families, including Intel Xeon M5zn instances, Graviton2-powered C6gn instances, Intel-powered D3/D3en instances, memory-optimized R5b instances and AMD-powered G4ad GPU instances. See InfoQ's coverage here.
There were announcements about Lambda and serverless deployments: billing granularity is reduced from 100ms to 1ms which reduces costs for every single Lambda function automatically, the availability of functions with up to 10 GB of memory and 6 vCPUs was added. One more new feature is the support for container images as a packaging format, to simplify the transition from current container-based workloads to serverless functions. To know more about AWS Lambda updates, see InfoQ’s article.
Adding Amazon ECS Anywhere and Amazon EKS Anywhere, AWS will make the container orchestration software used in ECS and EKS freely available to deployments outside AWS, including other cloud providers. This will provide increased integration and lower latency and will follow the path of Microsoft and Google that already offer Azure AKS and Google Anthos for free.
During the first keynote, the public preview of AWS Proton, a new managed deployment service for container and serverless applications, was announced. With AWS Proton, customers can automate and manage infrastructure provisioning and code deployments for serverless and container-based applications. See InfoQ's coverage here. ECR Public Repositories is a public container registry to store, manage, share, and deploy container images globally.
Storage
There were three major announcements for EBS, the block-storage service designed for use with EC2, including a new EBS gp3 volume type that is 20% cheaper than the previous gp2 type. Additionally, it provides a higher baseline performance and it is the first general purpose volume that allows to configure IOPS independently from disk size. Given the improvements and the simplicity of the upgrade, Corey Quinn, cloud economist at The Duckbill Group, suggests to switch immediately to the new volume type:
EBS gp3 is a game changer, full stop. 80% of the cost of gp2, it can be converted in place, and there's no downside. Do it immediately.
A new io2 Block Express volume type is available in preview, helping small-size high-IOPS workloads, and a tiered IOPS pricing for the io2 volume type is now in place.
The main update for object storage was that S3 now delivers strong read-after-write consistency automatically for all applications. AWS announced other improvements to S3, including replication with multiple destination buckets, two-way replication across regions to improve support for multi-master and multi-region applications and new bucket keys.
Database
There were important new launches for databases, including the previews of Babelfish for Aurora, a translation layer for Amazon Aurora PostgreSQL that enables Aurora to understand commands from applications written for Microsoft SQL Server. Aurora Serverless v2 is a new serverless relational database, compatible with MySQL, and AWS Glue Elastic Views builds materialized views that combine and replicate data across multiple data stores. There are separate coverages for both Aurora Serverless v2 and Babelfish for Aurora on InfoQ. Amazon Aurora PostgreSQL now integrates with AWS Lambda.
Different improvements and new features were introduced for Amazon Redshift, the data warehouse service, for example the ability to move clusters between availability zones, automatic table optimization and the previews of data sharing and support for native JSON data processing.
Networking and IoT
AWS Local Zones, single zone extensions of regions that are located near densely populated areas to provide lower latency, were introduced in 2019. During the conference, AWS announced that three new Local Zones will be GA this month in Boston, Houston, and Miami and twelve more in 2021, including New York City and Chicago.
Smaller Outpost options will follow next year as well, allowing customers to deploy AWS hardware in smaller offices, factories, and sites that are space-constrained but need access to low-latency compute capacity.
In the IoT space, the new AWS IoT Greengrass 2.0 provides an open source edge runtime and tools for local software development and managing software on large fleets of devices.
Machine Learning
Many machine learning new features and products were discussed during the keynote of Swami Sivasubramanian, vice president for AI and machine learning at AWS, and were focused around SageMaker. The new Amazon SageMaker Feature Store is a fully-managed, purpose-built repository to store, update, retrieve, and share machine learning features. See InfoQ's coverage here.
SageMaker Clarify, SageMaker Debugger, SageMaker Managed Data Parallelism and SageMaker Model Parallelism were other new services and features announced. Among many positive reactions, Jeremy Edberg, CEO at MinOps, highlights the benefits of Amazon SageMaker Clarify, the service focused on bias detection and explainability:
It helps you to detect biases in your dataset. I think it's fantastic; it's just surfacing that this problem exists. A lot of people don't even realize this is a problem at all. It's pretty amazing!
Coney Quinn finds the new approach confusing:
At launch, Amazon SageMaker was an easy onramp to machine learning for folks without formal data science training. Today, SageMaker Autopilot, SageMaker Studio, SageMaker Feature Store, SageMaker DataWrangler, SageMaker Ground Truth, SageMaker Notebook, SageMaker Neo, SageMaker RL, SageMaker Marketplace, SageMaker Experiments, SageMaker Debugger, SageMaker Model Monitor, and whatever they're released between the time I write this and the time I publish it mean that a neophyte is going to pull up the service page, get freaked out, shut their laptop and walk away.
AWS Trainium, a machine learning training chip designed by AWS and Habana Gaudi based EC2 built for machine learning were announced at the beginning of the conference, together with Amazon Monitron, an end-to-end system that detects abnormal behavior in industrial machinery, and AWS Panorama, a machine learning appliance and SDK.
Amazon Lookout for Metrics is one more addition in the anomaly detection space, adding a flexible service for time series analysis, together with Amazon Lookout for Equipment and Amazon Lookout for Vision.
Monitoring, Architecture and Coding
A significant part of Werner Vogels’ keynote was dedicated to the importance of logging, monitoring and how to improve deployments. A few new services and improvements are now available: CloudTrail provides more granular control of data event logging, while Amazon Managed Service for Prometheus (AMP) and Amazon Managed Service for Grafana (AMG) are now in preview.
AWS Fault Injection Simulator is a managed chaos engineering service that will be available in 2021 and introduces disruptive events across a range of AWS services, including EC2, EKS, ECS and RDS.
Cloudshell, a browser-based shell to interact with AWS resources, is already available and makes it simpler to work with the CLI without running an instance and handling credentials. Python is now supported by Amazon CodeGuru, the managed service for automated code reviews and application performance recommendations. Amazon Location, a service to integrate maps, location awareness, and other location-based features to web and mobile applications was announced in preview at the end of the conference.
AWS added more re:Invent sessions, for example on S3 topics, from January 12 to January 14, extending the conference to the new year.